DOCUMENT RESUME 



ED 372 630 



FL 022 323 



AUTHOR 
1ITLE 

INSTITUTION 
SPONS AGENCY 
PUB DATE 
NOTE 
PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Kim, Mi J. 

The MacMagic Program and Its Effects on "English as a 

Second Language" Students* An Evaluation Study. 

Beryl Buck Inst, for Education, Novato, CA. 

Marin Community Foundation, San Rafael, CA. 

93 

74p. 

Reports - Evaluative/Feasibility (142) 
MF01/PC03 Plus Postage. 

Classroom Communication; Comparative Analysis; 
'"'Computer Assisted Instruction; *English (Second 
Language); Grade 6; Grade 7; Intermediate Grades; 
Junior High Schools; *Language Arts; Language 
Proficiency; Middle Schools; Native Speakers ; Program 
Descriptions; Program Effectiveness; *Progra:n 
Evaluation; Second Language Programs; Social Studies; 
Student Attitudes 
-MacMagic Program 



ABSTRACT 

This report pressnts a description and evaluation of 
the MacMagic Program, a computer-facilitated core program in language 
arts and social studies in its fourth year of operation at Davidson 
Middle School in San Rafael, California. The program consists of two 
sixth- and two seventh-grade classrooms that use computers and 
cooperative learning techniques to enhance learning. The evaluation 
of the MacMagic Program sought, among other objectives, to assess the 
program's utility as a vehicle to enhance the learning of English as 
a Second Language (ESL) among the program's multilingual students. 
The evaluation consisted of classroom observations, administration of 
the Language Assessment Scale test to students, and comparisons of 22 
ESL MacMagic students with 22 similar students in the regular ESL 
program at the school. Overall, it found that students expressed 
positive opinions about ths MacMagic program and enjoyed working with 
computers. Comparisons between MacMagic and regular ESL students 
found no significant differences in test scores, absenteeism, and 
English language proficiency. The amount of social interaction 
between ESL students and native English speakers was higher in the 
MacMagic Program than in the regular ESL program. Recommendations for 
program improvement are presented. (Contains 102 references.) 
(MDM) 



Reproductions supplied by EDRS are the best that can be made t * 
w from the original document. 



o 

CO 
CD 

CM 
CO 

O 

LU 




BERYL BUCK INSTITUTE FOR EDUCATiON 

"Knowledge exists tO be imparted" Ralph Waldo Emerson 



The MacMagic Program and 
Its Effects on "English 
as a Second Language" Students 



November 1993 



An Evaluation Study Funded by 
The Marin Community Foundation 



J 



H 

J 




"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



U.S. DCPARTMINTOF IDOCATION 

Offict o* Educational R»Mtrch tnd Improvmul 

EDUCATIONAL RESOURCES INFORMATION 

CENTER (ERIC) 
\pjthts document t*«n f#pfOduc*d •» 
prec«iv«d from irv# carton or cwgamttlion 

originating it 
O Minor Chtngtft h»vt b«in mtd« to impfOvi 
'•production quality 

• Pomit ot vhiw ot optniont ttttftd in t hit doc u- 
mtnt do not n«c«uirily r#pr*Mnt otttCitl 
OERI po«itK>n or policy 



BEST COPY AVAILABLE 

2 



The MacMagic Program and 
Its Effects on "English 
as a Second Language" Students 



November 1993 



Ml J. Kim, Principal Investigator 
University of San Francisco 



Copyright © Mi J. Kim and Beryl Buck Institute for Education 1993 - All rights reserved. 



ERIC 




Beryl Buck Institute for Education 
1 8 Commtrciol Boulevard 
Novtto, CA 94949 

(415) 6834122 - FAX (415) 8&U&60 



Acknowledgements 



The. Marin Community Foundation funded this study on the differences in language 
acquisition between ESL students in the MacMagic class and in the other ESL classes at 
Davidson Middle School. The major work was conducted by Mi Kim, a graduate student 
from the University of San Francisco. 

Assisting Mi Kim with this study was the staff at Davidson Middle School, especially 
Principal Mary Buttler, Counselor Julian Podbereski, and the teachers and assistants from 
the MacMagic and ESL classes. There was also support from Jill Morgan and Dr. Edward 
de Avila in the analysis and interpretation of the LAS tests. The original draft for this study 
was written by Mi Kim. 

Dr. Colin Sacks conducted statistical analysis and contributed to Chapter VII. Dr. Carolyn 
Horan reviewed and revised the original document. Additional editing was provided by 
Matthew Stafford. 



Acknowledgements 



ERLC 



Purpose and Organization 



The purpose of this evaluation is to describe the implementation of the MacMagic program 
at Davidson Middle School in the San Rafael City Schools District and to assess its impact 
on ESL students* English language proficiency during the 1992-93 school year. 

Chapter I provides an introduction to the study. Chapter II describes the MacMagic classes. 
Chapter III describes the ESL classes (ELAC and Sheltered classes). Chapter IV describes 
the study's evaluation methodology, including its design, procedures, data collection 
process, and the instruments used. Chapter V describes the ESL students* perceptions of 
the MacMagic classes. Chapter VI discusses the student/teacher interactions in MacMagic 
and regular ESL classes. Chapter VII covers the results of quantitative analyses of 
Language Assessment Scale test data. Finally, Chapter VIII offers conclusions and 
comments. 
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Chapter I: 
Introduction 



The information age has revolutionized the way people live, think, interact and learn. In 
today's schools, computers are considered a vital part of education. Accordingly, numer- 
ous educational programs nationwide are incorporating technology into the curricula As a 
growing number of computers find their way into our schools, a series of fundamental 
questions regarding their role and effectiveness have been raised. 

There have been several studies dealing with the effect computers have on native English 
speakers' academic achievement (Dunkel, 1990). Yet only a few studies have been 
conducted to investigate the effect of computers on ESL students and their English 
acquisition and academic achievement. A series of questions on this topic was formulated 
by Dunkel to promote further study. 

■ Which kinds of computer-based lessons augment the development of 
particular English-as-a-second-language skills (e.g., reading and 
listening comprehension, oral proficiency, knowledge of grammar)? 

■ Which kinds of computer environments (computer-assisted instruction; 
computer-managed instruction; computer-enriched instruction) aug- 
ment the use and learning of English as a second language? 

■ Do students regard computer use as beneficial in improving their 
English language skills? 

■ Does small group work at a computer terminal generate conversational 
interactions among group participants? If so, what in particular is 
discussed? What is the quality of the discussion? 

■ Do certain features in the design of Computer-assisted Instruction 
(CAI)/Computer-assisted Language Learning (CALL) courseware (e.g., 
focus of control, screen composition) affect the quantity and quality of 
student learning? 

■ Can computerized-adaptive testing be used effectively and efficiently 
to assess reading and listening comprehension in ESL? 

■ Does the writing of Limited English Proficient (LEP) students improve 
as a result of learning and using word processing in English? 
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■ What types of CAI programs should be incorporated into the 
curriculum for LEP students? Should instructional CAI programs (e.g., 
tutorials, drill-and-practice, holistic practice, modeling, adventure 
reading) be used? Should collaborative CALL programs (e.g., 
simulations and games, annotation, modeling, and discovery 
programs) be used? Should facultative CALL programs (e.g., word and 
idea processing programs with spell check and on-line thesauruses) 
be used by older LEP students? 

These are just a few of the unanswered questions concerning the effectiveness of 
educational technology and ESL instruction. 

Today's classrooms increasingly reflect our multicultural, multilingual society. The number 
of students whose first language is not English is increasing rapidly, and teachers of 
bilingual education and ESL students have been searching for more effective ways to teach 
their students. What impact do computers have on multicultural, multilingual classrooms and 
students? 

This study examines a computer facilitated core program in language arts and social studies 
and its potential as an English-acquisition tool for the program's ESL students. The 
program, using computers as an integral learning tool, is called MacMagic and is in its 
fourth year at Davidson Middle School in San Rafael, California. MacMagic has a student 
body of high, middle, and low achieving students as well as multicultural, multilingual 
students in grades 6 and 7. The students are encouraged to work cooperatively on projects 
and to assess one another's work critically. This program provides a unique opportunity 
to examine the students and their progress in second language acquisition in the context 
of a technologically-based, multicultural classroom. 



Chapter I: Introduction 

12 



Chapter II 



Desccription of MacMagic Classes 



13 



Chapter II: 

Description of MacMagic Classes 



The computer-based core program, MacMagic, started as a joint partnership project of the 
Marin Community Foundation, LucasFilm, Apple Computer, and the San Rafael City 
Schools, The program consists of two sixth and two seventh grade core classes at Davidson 
Middle School. The program follows the California State Curriculum Framework for social 
studies and language arts and is similar to the other sixth and seventh grade core classes 
at Davidson, However, in the MacMagic classes, technology is a component of the 
curriculum, and it is used as an instructional tool for learning other skills. The objective of 
the class is to facilitate the students' learning through technology and cooperative learning. 

The program is located in two adjacent classrooms in a building isolated from the main 
school. One of the rooms contains a well-equipped Macintosh computer lab with such 
technologies as video editing machines, scanners, laser discs and CD ROMs, The second 
room is a standard classroom used for traditional class activities, group activities and 
discussions. Each classroom has two teachers, and there's one full-time technical support 
person in the computer room. 

The 1992-93 school year was the fourth year of the program. Students in the MacMagic 
program are selected to represent different learning abilities, cultural and ethnic back- 
grounds, and those whose first language is not English. The sixth grade MacMagic classes 
have 56 students, 10 of whom are ESL students. The seventh grade classes have 56 stu- 
dents; 15 are ESL students. These numbers reflect the overall percentage of ESL students 
in the middle school. Student selection was based on a formula designed to represent all 
segments of the Davidson School population and was administered by the school counselor 
and principal. 

The objective of the MacMagic program is to develop an innovative approach to teaching 
English and social studies, in which the teacher is viewed as a facilitator, rather than as a 
dispenser of information. The key element of this program is the integration of the 
computer to promote cooperative learning among students of diverse backgrounds. The 
students are constantly involved in group projects and are encouraged to participate in 
group activities. 

The computer-based core program is methodologically different from the traditional 
classroom, and the teacher-facilitator devotes a great deal of effort and time to preparing 
and designing the curricula. 

Teacher planning and cooperation provide the foundation for a program 
based on two apparently discrepant yet complementary principles: 
heterogeneity and teamwork. Guiding diverse groups of children to share and 
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collaborate in a challenging academic setting requires thoughtful attention to 
process. Accordingly, MacMagic Program meets daily to outline, develop and 
review the content of instructional activities and to select among procedures 
that will engage students in an innovative learning task. (Mergendoller, 1991) 

To ensure that all students would receive equal access to the curriculum, the teaching staff 
developed a framework that: 

■ identified and specified the elements of the core curriculum; 

■ adapted the curriculum to reflect the teaching philosophy of the 
MacMagic program; 

■ modified lessons to meet the needs of individual students; 

■ used technology to support individual students in their study of the 
core curriculum; 

■ used technology as the basis of writing, illustrating, and integrating 
language arts and social studies; and 

■ used technology as part of the learning process, not as a separate 
skill. 

MacMagic classes use three basic software packages: MacWrite, MacPaint, and HyperCard. 
Staff Description 

The teachers and the technical support person are crucial components of the program. 
Each MacMagic class has two teachers who collaborate as a team. 'Team teaching" often 
consists of dividing up the responsibilities or tasks. For example, one teacher might take 
charge of class discussions on readings while the other would be the "main" teacher in the 
computer room. These responsibilities switch back and forth. Sometimes, while one teacher 
is talking to the class, the other participates as a non-teaching partner, bringing up 
important points during the lecture, answering individual students' questions, or keeping a 
watchful eye on the students' behaviors. 

Besides the two teachers, at the sixth and seventh grade levels there is a technical support 
person who assists with the computer hardware and software. The technical support person 
also teaches such skills as how to create HyperCard stacks, buttons, fields with visual aids, 
flow-charts for the bio-stacks, and other computer-oriented subjects. She attends to 
students' individual needs whenever they have questions. The teachers and the technical 
support person work together, sharing their ideas and expertise through daily and weekly 
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meetings- The classes also utilize eighth grade teaching assistants from last year's 
MacMagic classes. 

Classroom Description 

The classes meet daily throughout the school year during three period blocks. The time the 
students spend in the MacMagic class each day is 2 hours and 25 minutes. Classroom 
activities revolve around reading, writing and social studies (e.g., world civilizations). 

The students usually sit in groups of four or five. The seating arrangement is changed four 
or five times over the course of the year by the teachers. The students work as cooperative 
units, with group assignments, projects, or presentations. The teachers consider the 
students' learning abilities as well as their language proficiency and social skills when 
assigning them to a group to encourage cooperative learning. Students are encouraged to 
take an active role in their own learning, to contribute to the group, and to help other group 
members. Frequently, the students are given assignments which involve decisions on 
organization, division of tasks among the members, and decisions on the presentation 
methods and styles. 

In the MacMagic Lab the students choose their seat/computer as they please, sitting next 
to whomever they choose. The students make independent decisions about involvement 
or interaction with others in the lab. Each student has access to a Macintosh computer. 

At the beginning of each class, the teacher goes over the students' schedule for the day, 
outlining what's expected to be finished by what deadline. Before the students begin their 
activities, the teacher explains in detail the assignment or activity, making sure that everyone 
understands it. Writing is an essential part of MacMagic. With computers accessible to every 
student, all writing activities go through word-processing. Almost every assignment and 
activity involves writing at some point. 

The program employs portfolio assessment techniques which include processes as well as 
products. There is no formal testing or quiz on the subject matter, with the exception of a 
weekly M word bank" test on the vocabulary. The students are graded on their homework, 
class assignments, group presentations, and individual presentations. They also submit 
work in progress on designated due dates. Students are evaluated by the quality of their 
computer work and by completed reports prepared with the word processing software. 
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Chapter III: 

The Transitional English Classes: ELAC 
and Sheltered English Core Classes === 

(Sheltered Core Instruction is in English and Social Studies) 

The comparison group students who participated in this study were enrolled in transitional 
English as Second Language (ESL) classes. There are three types of ESL classes at 
Davidson Middle School, ELAC 1 (English Language Acquisition Class 1), ELAC 2, and 
Sheltered English. ELAC 1 is for beginning ESL students who usually have been in this 
country for less than one year. ELAC 2 is for more advanced students. The Sheltered 
English Core class is for students who are more advanced but not quite ready for 
"mainstreaming." The focus is on subject matter content in English and in social studies. The 
ESL students enrolled in MacMagic would have been assigned to one of these ESL core 
classes if their names had not been selected randomly for MacMagic classes. 

The comparison group students in the study come from four different ESL classes: one 
seventh grade ELAC 2 class, one seventh grade Sheltered English Core class, one sixth 
grade ELAC 2 class, and one sixth grade Sheltered English Core class. The sixth and 
seventh grade ELAC 2 classes meet daily for three periods. The sixth and seventh grade 
Sheltered English Core classes meet for two periods. There were at least 20 students 
enrolled in each ESL class during the 1992-93 school year. Most classes have over 30 
students. The students are predominantly Spanish speaking, but there are also students 
who speak Spanish, Hindu, Akkan (an African language), French Creole, Vietnamese, 
Chinese, and other languages. 

The curricula for all of the 'ESL classes follow the California State Curriculum Frameworks. 
The instruction and the content is presented in a "sheltered" version incorporating frequent 
questions and answers that provide students with various examples and using different 
comprehensive strategies. Through this simultaneous exposure to content area instruction 
and English, the students work toward acquiring sufficient subject content and English 
language skills to be mainstreamed into the regular non-ESL classes at the school. 

Staff Description 

There is one teacher and one instructional assistant assigned to each class, with the 
exception of the seventh grade Sheltered English Core class, which has an instructional 
assistant for only one period. While the teachers have the main instruction responsibilities, 
the assistants help the teachers with individual student tutoring, class management, 
evaluation of students' work, and in some cases conducting content instruction. The role 
of the instructional assistants varies from class to class according to the teaching and class 
management style. The teacher and instructional assistant usually work as a team. 
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In some classrooms, teachers have students from more than one grade level and with 
different abilities and different language proficiency levels. The ESL teaches are frequently 
sources of information and support in helping students adjust to life in a new country. They 
also play the role of counselor and coach, giving attention to students who may be going 
through a difficult socio-economic, linguistic, cultural, or family situation. Teachers and 
instructional assistants provide students with individual attention during class, during lunch 
period, and after school whenever possible. 

Class Description 

The classes meet daily over the entire school year during three period blocks (two periods 
for the seventh grade Sheltered English Gore class). The time students spend in a three- 
period block class is 2 hours and 25 minutes. Classroom activities revolve around language 
arts and social studies. The most important aspect of the class is the instruction of English 
as a second language. 

There is one Macintosh Classic computer in each of two ESL classrooms, as well as an 
overhead projector used for class instruction. ESL teachers use different instructional styles, 
methods and approaches. For example, in one class the students sit in groups of three or 
four, while, in another class the students sit in rows in teacher-assigned seats, which remain 
unchanged throughout the year. 

In classes where the students sit in groups, they often work as cooperative units on group 
assignments, projects, activities and frequent decision-making discussions. There is frequent 
interaction between the students, the teacher and the instructional assistant. Few behavior 
or discipline problems have been observed in the classes employing cooperative learning 
groups. In classes where students are seated i 1 traditional classroom rows, the teachers 
go over the instructional materials and students are expected to work on assignments 
individually. Verbal and non-verbal interactions are not encouraged. However, it's second 
nature for students to ask someone nearby to explain an unclear instruction. In many cases, 
the students move around during the class to talk with someone across the room. Much 
of the class time is spent on control of the students' talking, physical activities and 
movements. 

In both types of ESL classes, the teachers employ Sheltered instruction techniques, 
involving frequent questions and answers to allow students and teachers to negotiate the 
meaning of words and concepts in English. By repeating key words and phrases and 
through slow speech, the use of controlled vocabulary and expressions, the use of visual 
aids and an extensive use of body language, the teachers provide a variety of opportunities 
for students to learn the material. 

As noted above, class management styles are different from one teacher to another. 
Regardless of approach, the atmosphere is non-threatening and supportive. Students are 
encouraged to express themselves in English and teachers do not necessarily correct every 
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mistake the students make. Corrections are kept at a minimum as long as the mistakes do 
not interfere with communication itself. 

Typically, the teacher first introduces students to a literature piece, focusing on vocabulary. 
The teacher leads the students in a discussion of the topic. Then individual or group 
assignments are given related to the topic. In addition to language acquisition, the ESL 
class also spends time on the instruction of English grammar and basic vocabulary studies. 
The students have regular assignments and tests on the grammar and vocabulary that they 
learn each week. 

The students in the ESL classes have the same projects as other non-ESL language arts 
and social studies core classes, such as a Greek project, an Egyptian project, a report on 
a country for the sixth graders, and research reports and other projects for the seventh 
grade. Besides the special projects, the students have regular assignments and homework 
on reading, writing, spelling, grammar and vocabulary. 
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Chapter IV: 

Evaluation Methodology 



Student Demographies 

All the non-English or limited English speaking students in the MacMagic classes and a 
matched group of students from the regular ESL classes were selected to be part of this 
study. In the following section, the student demographics information is presented on three 
tables. Table 4.1 illustrates Class Composition of the MacMagic Rrogram, the number of 
ESL students in the MacMagic Program and the number of ESL students who were in a 
regular ESL program. Table 4.2 identifies the number of students who are considered either 
ELAC or Sheltered (*) students in both MacMagic and in the regular ESL program at 
Davidson. Tables 4.3 and 4.4 illustrate the Native Language Background, Sixth and Seventh 
Grades respectively, for the study group (MacMagic students) and the comparison group 
(regular ESL program student^,. 



Table 4.1: Class Composition (Gender/Grade) 



Group 


Sixth 




Seventh 




MacMagic Program 


Total 


59 


Total 


56 


(Total # of Students) 


Boys 


27 


Boys 


27 




Girls 


32 


Girls 


29 


MacMagic ESL 


Total 


7 


Total 


15 


(Study Group) 


Boys 


3 


Boys 


9 




Girls 


4 


Gihs 


6 


Non-MacMaglc ESL 


Total 


9 


Total 


17 


(Comparison Group) 


Boys 


5 


Boys 


11 




Girls 


4 


Girls 


6 



* Sheltered students are designated limited English proficient and are 
preparing to enter regular education classrooms. 
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Table 42: English Proficiency Level 





Ena Proficiency 


Sixth 


Eng. Proficiency 


Seventh 


MacMaglc 

(Study Group) 


ELAC 11 
Shattered 


5 
2 


ELAC II 
Sheltered 


1 

14 




Total 


7~~ 


Total 


is" 


ESL 

(Comparison Group) 


ELAC II 
Sheltered 


6 
3 


ELAC 11 
Sheltered 


3 
14 




Total 


9~~ 


Total 


17~ 



ELAC: English Language Acquisition Class 



Table 4.3: Native Language Background, Sixth Grade 



Study Group 

Spanish 6 
Vietnamese 1 
Chinese 0 


Comparison Group 

Spanish 7 
Vietnamese 1 
Akkan 1 


Total 7 


Total 9 



/ 



Table 4.4: Native Language Background, Seventh Grade 



Study Group 

Spanish 8 
Vietnamese 2 
Chinese 1 
French Creole 2 
Turkish 1 
Polish 1 


! 

Comparison Group 

Spanish 15 
Vietnamese 0 
French Creole 1 
Persian 1 


Total 15 


Total 17 
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Evaluation Design 

Teachers and students were observed on a regular basis at least twice a week in order to 
assess the students' language interactions and behaviors in the computer-based class over 
the course of one school year. Qualitative data were also collected from taped and untaped 
interviews with the teaching staff and students. The purpose of the observations in 
MacMagic classes and the comparison ESL classes was to monitor students' language 
interaction among themselves and with the teachers, and to document the language 
progress of the ESL students. The observations occurred mainly in the MacMagic classes. 

The Language Assessment Scale (LAS) test, administered before and after the class, 
provided individual scores for oral language and writing. The test was administered in late 
September and late May to the study group (MacMagic classes) and the comparison group 
(ESL: ELAC/Sheltered classes). At the same time taped interviews and writing samples from 
the students were gathered. 

Permission letters were sent to the students' parents or guardians, and only those students 
who had approval were included in the study. 

Student Population 

Ten sixth grade and 15 seventh grade MacMagic students were in the study group. An 
equal number of students was selected for the matching comparison group. 

The ESL students in the MacMagic classes were randomly selected from a list of students 
at the end of the 1991-92 school year. The comparison group was selected from approxi- 
mately 100 sixth and seventh grade ESL students in the school. (Forty-six students in the 
sixth grade were classified as LEP/NEP students. Eight of them were in the MacMagic 
classes. Fifty-four students in seventh grade were classified as LEP/NEP students, and 15 
of them were involved in the MacMagic classes.) With the cooperation of the teachers and 
school counselor, 26 students were matched to the MacMagic students to make up the 
comparison group. 

In the process of selecting the students for the comparison group, the following criteria were 
applied as closely as possible. Students were matched for: 

■ grade level and age; 

■ academic performance level as identified in IPT (IDEA Oral Language 
Proficiency Test) and LAS (Language Assessment Scale) scores; 

■ English proficiency level; 

■ gender; 
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a native language background; and 



■ teacher perception. 

Due to the lack of adequate background information on the students, it was difficult to apply 
all of the above criteria in every case. While the teachers' input was the most valuable 
criterion, grade average, attendance record, and relevant academic history in the students' 
files was used in deciding which students would form the comparison group. It was not 
possible to find a perfect match because of the small number of students in the available 
pool. As a group, the ESL students in MacMagic had a higher language proficiency than 
the ESL students in the comparison group. 

Data Collection Process 

Data were collected throughout the 1992-'93 academic year. Non-participant and participant 
observations and interviews were used to obtain an accurate description of the classes. The 
focus of the observation was the language interaction between the students and the 
teachers and among the students themselves. The students' computer interaction was also 
observed. Pre- and post-tests on English language proficiency were given using the 
Language Assessment Scale (LAS) Oral and Reading/Writing. 

Interviews 

Taped and informal interviews were conducted during the regular classes of the MacMaflic 
program. Participant and non-participant conversations and dialogues were employed to 
gather data on the ESL students' perceptions of the class, attitudes about using computers, 
and students' own interpretations of their interactions with others. Informal student interviews 
offered much data on student interaction, use of language, and cultural perceptions. 

Informal interviews with teachers in the MacMagic and ESL classes were conducted in the 
form of "lunch room chatting" and "sharing ideas." This provided a greater understanding 
of the teachers' perceptions of the classes and their dynamics. 

M jst of the interviews were conducted in English. A few with students, who weren't able to 
express their ideas in English adequately, were helped in interpretation by students who 
spoke the same language. Since Spanish was the predominant first language among the 
ESL students, a Spanish speaking teaching assistant provided additional interpretation for 
those interviews. This helped to reduce the student's interview-related stress. 

The taped interviews were transcribed and the major themes were identified. The informal 
interviews were jotted down as part of the observation field notes, which were transcribed 
later. 



9 

ERIC 



4.4 



Chapter IV: Evaluation Methodology 

25 



Observation 



Classes were observed throughout the school year. The frequent classroom presence of the 
observer (one to three times per week) reduced the problem of student reactivity. The 
open, flexible nature of the MacMagic classes was another asset: Many people, including 
teachers, technical support personnel, and the students themselves, frequently walked in 
and out of the classrooms, and there were many observers from other schools, the district, 
or community organizations. For that reason, the students and the teachers were 
accustomed to having visitors in their classes. 

Observations concentrated on class dynamics and on the natural interactions, reactions, 
and progress of the ESL students. One focus was the students' language interactions, 
especially the interaction between the ESL students and the teacher, the ESL students and 
other native students, and among ESL students themselves, as well as the students' 
interaction with the computer. Notes were taken throughout the observation and were later 
reviewed and their major themes identified. 

Student Assessment Instrument 

The state-approved Language Assessment Scale (LAS) was used in this study to assess all 
ESL students in both the computer-based core classes and the ESL core classes. The 
selection of the LAS was based on consideration of validity, reliability, cost, and ease of 
administration. 

LAS-Oral: Forms 1C/1D, 2C/2D: 

Language Assessment Scale-Oral assesses the students' oral language skills. Forms 1C 
and 1 D are used for sixth graders and Forms 2C and 2D are used with seventh graders. 
The LAS-0 contains three sections: Vocabulary, Listening Comprehension, and Story 
Retelling. The LAS-0 is usually used to test students' oral English proficiency level for 
placement purposes. 

Forms 1 C and 2C were used for the pre-test and Forms 1 D and 2D were used as the post- 
test. As noted, the LAS-0 has three sub-components: Vocabulary, Listening Comprehen- 
sion, and Story Retelling. 

The Vocabulary Section tests the student 's ability to produce words to go with the pictures 
shown by the test administrator. Forms 1C and 1D have two sub-sections under the 
Vocabulary section. The "Name that Picture" section tests the student's ability to produce 
labels for objects in the picture shown, and the "Action Words" section tests the student's 
ability to come up with the action verbs shown in the picture. Forms 2C and 2D also have 
two sub-sections, "Action Words" and "Opposite Words." In the "Opposite Words" section, 
the student's ability to produce an antonym to a given word is tested. 
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The Listening Comprehension Section measures ths student's listening comprehension 
ability. The student listens to a taped conversation while looking at the cue pictures 
provided. Afterwards, the student responds to a series of true-false questions. 

The Story Retelling Section gathers oral language samples of students so their oral 
language proficiency can be holistically rated. The student listens to a taped story while 
looking at the provided cue picture book. Afterward, the student is asked to retell the story 
in his or her own words. The examiner writes down the student's response while recording 
it on tape. The complete response is transcribed. Later, the "Story Retelling" section is rated 
according the guidelines provided by the LAS manual. The scores are rated holistically on 
a scale of 0 to 5 by two or more raters. The "Story Retelling" section is the biggest part of 
the lAS-0 score. 

LAS-Reading/Writing Form 2A/2B, 3A/3B 

The LAS-Reading and. Writing assesses the student's ability in these areas. The LAS- 
ReadingAVriting has two forms, A and B for each level. Forms 2A and 2B are used with the 
sixth grade, Forms 3A and 3B with the seventh grade. Forms 2A and 3A are used as the 
pre-test while Forms 23 and 3B are used as the post-test. 

The LAS-RAV level 2 has seven sections, including Vocabulary, Mechanics and Usage, 
Fluency, Reading for Information, Finishing Sentences, What's Happening? and Let's Write. 
The LAS-RAV level 3 has seven sections as well: Synonyms, Fluency, Antonyms, Mechanics 
and Usage, Reading for Information, What's Happening?, and Let's Write. 

Vocabulary measures the sti "Jent's ability to match pictures to words at levels 1 and 2. 

Svnonvms measures the student's ability to identify words with similar meanings at level 
3. 

Antonyms measures the student's ability to identify words of opposite meanings at level 3. 

Fluency measures the student's overall language fluency and the ability to infer a missing 
word based on knowledge of language usage and semantics. Items of appropriate difficulty 
are included at all three test levels. 

Mechanics and Usage measures the student's skills in capitalization, punctuation, and 
grammar usage at all three test levels. 

Reading of Information measures the student's ability to gather information from written 
material. Each level contains one story and 10 test items, which at level 1 are true-false 
questions, and at levels 2 and 3 are multiple choice. 
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Finishing Sentences measures the student's ability to complete a sentence correctly with 
five open-ended test items at levels 1 and 2. 

What's Happening? measures the student's ability to write an original sentence. At each 
of the three levels, it includes five graphic prompts; the student writes a sentence about 
each one. There is a three-step procedure for scoring the "What's Happening?" section as 
given in the LAS Scoring and Interpretation Manual. 

Let's Write measures the student's ability to write a story. Level 2 uses a graphic prompt, 
and level 3 uses a verbal prompt. There's a scoring guide for the "What's Happening?" 
section as given in the LAS Scoring and Interpretation Manual. 

Oral Language Sample 

The Story-Retelling section from both the pre-test and post-test of the LAS-0 was separately 
evaluated. More detailed holistic rating of the speech sample was performed according to 
the provided Oral Language Sample rating system on the scales of 0 to 5 with a plus in 
each level (0, 1, 1+, 2, 2+, 3, 3+, 4, 4+, 5). The LAS scoring system has a scale of 0 to 
5, but with the LAS scoring system, the majority of the students tend to fall under the rating 
of 3, since 3 is the middle rating. There are very few ratings of 4, 5, or 0, 1. 

The length of the oral speech sample in the pre-test and post-test was examined to 
determine any gains in terms of length. 

Writing Sample From LAS-R/W "Let's Write" Section 

Writing samples gathered as part of the LAS pre/post test were separately analyzed with a 
more detailed rating system to measure the fluency in written language more closely, 
independent of other LAS Reading/Writing section scores. The length of the writing samples 
was also examined to investigate the overall gains in terms of the quantity of writing. 

Items Included in the Evaluation 

The following questions formed the basis for data collection: 

1 . How do ESL students perceive the MacMagic class? 

2. How do the ESL students interact with teachers, native speaker 
students, and among themselves in the MacMagic class? 

3. What do the ESL students do in the MacMagic class? 
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4. How much gain will the MacMagic ESL students achieve by the end 
of the school year as measured by the Language Assessment Scale 
(LAS)? 

a, How much gain will be achieved on the general LAS 
scores? 

b, How much gain will be achieved on the fluency of the 
writing samples from the LAS in terms of holistic scoring 
and average length? 

c, How much gain will be achieved on the fluency of the 
oral language sample from the LAS in terms of holistic 
scoring and average length? 

5. What gains will the non-MacMagic ESL students achieve by the end 
of the school year as measured by the LAS? 

a. How much gain will be achieved on the general LAS 
scores? 

b. How much gain will be achieved on the fluency of the 
writing samples from the LAS in terms of holistic scoring 
and average length? 

c. How much gain will be achieved on the fluency of the 
oral language sample from the LAS in terms of holistic 
scoring and average length? 

6. Are there any differences in English language proficiency achievement 
scores at the end of the school year between the MacMagic ESL 
students and non-MacMagic ESL students from other ESL core 
classes? 

7. Is there any difference in attendance between the MacMagic ESL 
students and non-MacMagic ESL students? 

In order to answer questions one through three, notes from observations and interviews 
were reviewed to bring out common themes. 

To answer questions 4 through 6 the general overall scores of the ESL students' pre-test 
and post-test were compared to determine any gains. The holistic scores of the writing and 
oral language samples from the pre- and post-test "Let's Write" and M Story Retelling" 
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sections were compared. Also, the average length of the writing samples and the oral 
language samples were examined to determine any gains in length. 

To answer quer+ion 7, the student attendance records were obtained, and the mean 
reported absence of the ESL students in the MacMagic classes and the ESL core classes 
were compared. 

Calendar of Events in Carrying Out the Study 

Both the computer based core class group and the comparison ESL class group were pre- 
tested in September/October and post-tested in May/June. Between the pre/post-tests, the 
classes were closely observed at regular intervals. After the instrumentation was completed 
and the results from the pre-test, post-test, and writing samples were gathered, the study 
analysis was performed. The study Schedule is presented in Table 4.5 beiow. 



Table 4.5: Schedule 



Sept. Oct. Nov. Dec. Jan. Feb. Mar. Apr. May June 

I — I 

Pre-test Post-test 

Writing sample > Observation < : — Writing sample 



Limitations of the Study 

Eight different classes were involved in this study. While the MacMagic classes had the 
same teachers and technical assistant, the comparison group of ESL students came from 
ESL classes with four different teachers and instructional assistants who had distinctive 
teaching styles and class management practices. 
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Chapter V: 

ESL Students' Perceptions of 
MacMagic Classes 

Overview 

In general, the students expressed positive opinions about MacMagic. They seemed to 
appreciate the free atmosphere, low stress levels, and the relaxed classroom environment. 
They felt that MacMagic classes placed the responsibility of learning on the students 
themselves. Students are in charge of their own learning processes, and each student can 
learn and work at his or her own pace. Teachers were seen as learning facilitators rather 
than knowledge dispensers. 

MacMagic seemed to raise students' interactions with one another as well as the level of 
teacher attention each student received. Although many ESL students in MacMagic have 
been in the United States less than four years and lack the English proficiency to be 
mainstreamed, they contributed as members of their respective groups in activities with 
students who were native English speakers. 

The ESL students had positive comments about class computer use. Each student identified 
computers as a unique, interesting, fun feature of MacMagic. In this chapter, MacMagic 
student comments are presented. 

On Computers 

The computer-aided benefit most mentioned by students was the ease of the editing 
process. Computers allow them to add or delete text and to reorder entire paragraphs or 
sections. Other choices available to students working with computers include type of font, 
graphics, sound and animation. 

Sample comments from the ESL students include: 

"I think it is neat how computers function, it makes work a lot easier." 

"I love it, I just like working on computers. It's fun. If I make a mistake, I don't have 
to start all over again, it's a lot easier to correct the mistakes I make. So I get to try 
many different things." 

•I like to use HyperCard because I can create a lot of interesting and fun, really nice 
stacks." 
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"If I make a mistake on a paper, I have to rip it and start ail over again. But with the 
computer, you just delete or correct your mistake on screen with a few keys, it's so 
simple. You can also use spell check to help you." 

"With the computer, you can do many different things like animation with paint tools 
and HyperCard." 

"You can learn anything you want to learn with the computers in this class." 

"You don't have to write with your hands anymore. You just do everything on the 
computer. It's easier-it doesn't hurt your hand to write a lot." 

"Because you get the printout of your work, writing, or drawing, it looks really nice. 
You have very nice looking prints out of your work, also it is so easy to correct the 
mistakes and change them." 

"I like working on the computers because I can get a job working on the computers 
later when I grow up because I know how to use the computers." 

"Computers are smart and they tell me what to do and how to think about things." 
On Group Work 

One of the most important features of MacMagic classrooms is the integration of curriculum 
and technology with the cooperative learning strategies. The student-centered group 
activities are enhanced by multi-media/hypermedia tools. The students work in groups of 
two to four. The cooperative groups change four to five times throughout the year to 
promote interactions among a wide range of students. Teachers recognize the positive 
effects on social as well as academic levels. Typical cooperative activities would involve 
group projects requiring social interaction, learning through discussions, questions, 
negotiations, compromises, and creations. Students frequently participate in peer editing 
and peer tutoring. The students appeared to recognize the benefits of cooperative learning 
groups as they repeatedly mentioned "group work" as one of the unique features of 
MacMagic. Sample student comments follow: 

"I like working with other people because when I don't understand 
something, one of them always knows the answer and can help me." 

"Sometimes, it's not fun working with others because some people don't 
work hard; they don't cooperate and the rest of us end up doing all the work 
for the person, it is not fair." 

"We also have class meetings and I really like that because we can tell the 
teachers what is going on and how we feel." 
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"1 like working with people who are smart, not with those who just fool 
around." 



"The most interesting thing in MacMagic was computers and working with 
friends." 

"In other classes, I have to just sit at my desk, but in this class I sit with 
others and we can also move around in the classroom. 1 ' 

On Differences Between MacMagic and Other Classes 

The ESL students in MacMagic noted the following differences between MacMagic and 
other classes: 

"We use computers [in MacMagic]. We draw and write on the computers 
too." 

"We have less homework [in MacMagic] but more work during the class." 

"In other classes, I don't have to stand in front of the class and talk. I have 
to in this class." 

"In my science group, we also work in groups but there are many Spanish 
speaking people, but in this class, there are not too many." 

"We stay in MacMagic for three periods while other classes have only one 
period." 

On Learning English 

The students' self-perception comments reflected the nature of the impact of MacMagic on 
the students' second language learning process. Eighteen out of 22 MacMagic ESL 
students thought their oral language skills were better than their written language. Twenty 
out of 22 expressed the desire to improve their writing skills. The most frequently 
mentioned difficulty in English was vocabulary. The ESL students have internalized the 
importance of vocabulary in learning English. To ESL students, knowing the meanings of 
words and their spellings was considered one of the most important parts of learning 
English. Following are some typical comments by MacMagic students. 

"Sometimes, when people talk, they have some complicated words I don't 
understand, then I can ask someone in the class." 

"This class is all about learning English and reading language. In other 
classes, we dop't do too much reading. I get to communicate more, and get 
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to talk more. Sometimes, I hear a word which I don 't know but I get to listen 
to others using the word and learn the meaning." 

■This class helps me learn more words." 

■I learn more English in this class than other classes because we in this 
class, we have word bank, the teachers make us look up the dictionary." 

"[I learn some words by] Listening to what other Americans say and trying 
to figure what they are saying. I sometimes write them down to remember." 

" The native speakers help me when I don't know some words. They tell me 
their meanings." 

"I learn more words in this class because of the word bank. I have to know 
the definition and sentences for the test. I have to memorize them." 

"While working on the computer like the 4 ABC stacks, we have to talk to 
others to find out how to do certain things or how to spell some words in 
Spanish. I get to speak more with others in English." 

■ With the spell check, i find out my mistakes in spelling;" 

"I don't think I learn more English in this class because in my other classes, 
I always speak English with everyone (except my science class)."' 

"I learn more English in this class because I have to talk with people, doing 
my work with others. Here I learn more English in this class because we only 
study about English." 

•I ask questions to anybody if I don't understand something. I learn English 
by listening and talking to others. If I hear somebody speaking some new 
words I didn't know, I pick it up. And if I don't understand, I always walk up 
to the person and ask the meaning." 

"I think I learn more English here, I mean words, and reading stuff. Because 
it's longer than other classes. We have three periods." 

"You have to write a lot in this class. There are a lot of stuff to do and you 
have to go to the front of the class and speak." 

"I think I learn to speak English in this class because the teachers and other 
students only speak in English. I hear English all the time." 
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"The teachers speak a lot in English and they make sure that I understand 
them/ 

"I talk to other students a lot in this class. That helps me with my English 
speaking and listening/ 

On Motivation and Attitude in Second Language Learning 

Comments provided by ESL students suggest that they want to learn to speak English to 
function as members of society as well as to be able to get jobs later on. Typical comments 
follow: 

"Learning English is important because I want to communicate with others. " 

■I like to be able to write better. It is hard to describe what I want to say in 
English yet. I want to be a good writer too." 

"Sometimes, when the native speakers speak, they have some complicated 
words that I don 't understand. I want to know every word spoken by others." 

"It is very important to speak good English because if you don't speak 
English, you can't talk to anybody. That's really hard for you." 

"There are so many words in English. I need to learn them." 

"Learning English is important, because if I am learning another language. 
I can get a better job." 

"If you don't know how to speak English, kids pick on you. So you have to 
know how to talk back." 

"I like to read stories. I wish I could be a better reader." 

"Now I live in America and I have to learn their language. Sometimes I have 
to talk to Americans. If I can't speak English, I can't talk with them." 

"Going to school is important because I learn new things. And in order to be 
able to go to school and understand the teachers and others, I have to learn 
English." 

"If you know how to speak English, you can help others who don 't speak the 
language, my friends, my parents, and others." 
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"If you know two languages, you know more friends and you can have better 
work if you are bilingual." 

On Their First Language 

The ESL students also noted the importance of their [first] languages. Most of the students 
speak their first language at home with their parents, siblings and friends. Many students 
also speak their first language at school with students who speak the same language. 
Speaking a first language was frequently associated with students' cultural and ethnic 
identities, while speaking English was associated with learning and school. Sample 
comments follow: 



"I don't speak English too much. I like to improve my English, but I don't 
want to forget my first language." 

"I always speak in my first language when I speak to my family, relatives or 
other friends who know my language/ 

" Speaking Spanish is important because it is my language. And if I don't 
know how to speak Spanish I can't talk to my parents." 

"Speaking Spanish is important because I am a Latino and it's my culture. 
[They all speak Spanish.] If I go to my country and only speak English, I 
wouldn't know how to speak with 3thers. I go back to my country sometimes 
to visit. I wish I could go back to my country and live there." 

"Both English and Spanish are important to me, because I have to know and 
use both languages every day." 

"Speaking Vietnamese is important to me because I can understand what my 
mom and dad are thinking and speaking about. I can listen to them." 

"I speak Polish with my family. I have to speak the language to communicate 
with them." 



On Teachers 



The MacMagic students noted the different teaching styles of MacMagic teachers. What 
follows are some representative comments from the students about their teachers: 

"There are two teachers in this class, we only have one teacher in the other 
classes." 
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"The teachers try to help me as much as they can. They try to make things 
clear. They help me whenever I need their help." 

"Sometimes, it's hard to get the teachers' attention, especially in the 
computer room. I have to wait for a long time, when they are talking to other 
students/ 

"I feel very comfortable speaking to the teachers because they let you ask 
any questions. They are friendly. " 

"The teachers are nice and they will help you, They help you and they even 
make jokes. They are fun." 

"I always try to ask questions to the teachers rather than to other students 
because sometimes other students don't know what they are doing either. 
The teachers know the best." 

"The teachers are nice." 

"The teachers want you to have more fun with what you do and learn in Wm 
class." 

"The teachers in MacMagic let you decide oh many things when you are 
working on the projects, but the teachers in other classes always tell you what 
to do." 

"The teachers don't give us a lot of homework and they know that we like 
to work on the computers." 

"The teachers only speak in English so I hear English all the time." 

"The teachers help me a lot with my homework and with spelling. If I don't 
understand what a word moans, the teachers explain a lot to me. They also 
help me after school. I stay after school for two to three days per week but I 
don't mind. The teachers make sure that I understand all the homework." 

Students ' Suggestions for Improvements and Changes 

The MacMagic students also offered suggestions and comments concerning ways in which 
MacMagic could be improved. Others felt no changes were called for. 

"I wouldn't change anything about MacMagic. I wish I could be in eighth 
grade MacMagic just like this one next year." 
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"I wish there were more fun activities like a birthday celebration. I also wanted 
to introduce my country to other students. And others who came from other 
countries should do that too." 

"I think they [the teachers] should ask the students what they think about the 
projects or ask what kind of activity we would like." 

"I hope we could sit wherever we wanted." 

"I hope that the teachers would tell me earlier if I was missing any work so 
that I could make up." 

"We do the same thing like 'current events' and 'word bank' over and over 
again. 1 wish we could use computers for these activities as well." 

"They have 'word bank' and 'current event, ' it's O.K. for the first couple of 
months, but it gets boring. They should change that to something different. 
Most people in class don't like to do it because it's boring. They think it's 
the same assignment again and again." 

"I wish I could play 'word bank' games on the computers rather than doing 
the work on paper." 

"i like to spend more time on the computers than anything else." 

"The teachers should keep the disks because some kids go around and 
delete others' work." 

"I wouldn't let students sit next to his or her friends in both the computer 
room and the classroom because they will constantly telk and not pay 
attention to the teachers and don't do too much work." 

"I wish this class had more people who speak my first language. Because 
I can ask questions in my language if I don't understand something." 

"I don't like doing the reading record." 

"I wish they didn't make me speak in front of the class. I get very nervous 
when I have to do that. I like to be able to give my report in person privately." 

"I wish we could do less social studies and more language arts 
assignments." 
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Chapter VI: 

Student/Teacher Interactions in 
MacMagic and Regular ESL Classes 

MacMagic classes have a distinctive style. In this chapter, the general interaction patterns, 
especially the language interactions between students and teachers identified during year- 
long observations, will be discussed. 

Language Interactions in MacMagic 

The atmosphere in MacMagic classes is relaxed and easygoing. Students talk to each other 
constantly, asking questions, enjcving conversation and working with the computers. The 
students are not restricted to their desks and.they can walk up to anyone and ask questions 
in the MacMagic Lab. When they're in the MacMagic classroom they're more restricted in 
their activities. MacMagic classroom activities consist of discussions, group or individual 
presentations, reading, or group activities that do not require computers and other 
technologies. Teachers function as facilitators and students are encouraged to gather 
information from each other. In the MacMagic Lab students cannot ask questions of the 
teachers until they have asked at least two classmates. 

Once the groups of friends were established in the MacMagic Lab, seating patterns 
remained constant throughout the school year. The teachers and technical support person 
circulate around the Lab and provide assistance when a student requests help. 

The ESL Students in MacMagic 

There were 15 ESL students in the seventh grade and 7 ESL students in the sixth grade 
MacMagic classes. Out of 22 ESL students involved in this study, 14 students spoke 
Spanish as their first language. Other students' native languages were French Creole, 
Turkish, Polish, Vietnamese and Chinese. In all, there were 10 girls and 12 boys. 

The seventh grade morning MacMagic class included 10 ESL students, seven of whom 
spoke Spanish. The ESL English language proficiency level varied from student to student. 
The students with higher English proficiency appeared to interact more with English 
speakers in the class, participating in the class discussions as well as other activities. The 
ESL students with low English proficiency tended to be more reclusive and withdrawn. The 
students who knew others who spoke their language (predominately Spanish) tended to 
form their own support group within the class. They sat together and worked together in the 
MacMagic Lab. They spoke Spanish to one another, frequently explaining the tasks, asking 
and answering questions. There was one boy, A, who refused to speak English for one 
semester and ignored any English spoken to him. Another Spanish speaking boy, B, always 
helped him, translating others' speech, and explaining assignments and activities. They 
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always sat next to each other in the MacMagic Lab and worked together. Other Spanish 
speaking students helped wnenever they were around. In the Spanish speaking group boys 
and girls usually did not cross the gender line when they interacted. There were two 
Spanish girls who always sat together in the MacMagic Lab sharing their work, always 
talking in Spanish. However, they barely interacted with Spanish speaking boys. 

Those who didn't understand could count on their friends to explain things in Spanish. In 
many instances, there was bilingual instruction occurring between ESL students who spoke 
the same language. When students did not know enough about a subject to teach it to their 
friends, they usually asked others for help. Strong ethnic solidarity among Spanish 
speaking students was observed. 

Lone representatives of a native language tended to assimilate and interact with English 
speakers. At the beginning of the school year, ESL students in seventh grade classes 
tended to associate with each other, even though they did not share the same native 
languages. This might be because many of them had the same ELAC or Sheltered Core 
classes the year before. Perhaps they also felt more secure speaking with someone who, 
like themselves, did not speak English fluently. In sixth grade this phenomenon was not as 
strong. A form of ethnic solidarity among Spanish speaking students was observed, but the 
close association among all ESL students was not as apparent. This might be because 
they came from six different elementary schools. 

In some cases, racial or ethnic factors proved to be important variables. C, a Caucasian ESL 
boy, used to be friendly with D, a Spanish speaking ESL student. Both of them were at the 
advanced proficiency level. But as time passed, C identified more with other Caucasian 
English speaking boys, and eventually became quite good friends with one of them. D and 
C were still talking occasionally, but C identified with other non-ESL students. The students 
from Haiti identified more wth other African American students as time passed. They sat 
together when permitted and participated in the classroom activities together. 

The native culture of the students was another important variable in student interaction. For 
example, Asian students were pretty much on their own, and resisted asking questions 
either of the teachers or of others. One boy said, "Asking is not very good, I always try to 
find answers by myself," and another said, "You are not supposed to ask, if you don't 
know something, it's your fault." "I always ask the teachers if I have to, I don't like asking 
others [other students]/ Asian students, especially those who have been in the United 
States less than three or four years, tend to work by themselves if allowed. Even in the 
MacMagic Lab, where most students converse constantly, they have very few interactions 
with others. Unless they are assigned to a group activity, they always work individually. 
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What Do the ESL Students Talk About? 

ssess-e^^ *» »«"*- d ,he ,evei °* native ,an9ua9e 

proficiency . 

Types of Interactions 

& to the behavior. The interaction variables are as Mows: 

Giving an explanation 
Receiving an explanation 
Receiving a response without an explanation 
Receiving a response to a procedural question 
Giving short answers 
Receiving no response 

Lee (1989) generated Peer Interaction Categories incorporating Webb/s factor. 
Sables ! HeMnteraction Categories are classified into four m a,or areas of interacts. task- 
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related interaction, procedure-related interaction, socio-emotional interaction, and miscellan- 
eous off-task interaction. 

Lee's categories are as follows: 

1) Task-Related Interaction 

•Gives task-related help 

•Gives inadequate task-related help 

•Asks a question and receives task-related help 

•Asks a question and receives inadequate task-related help 

2) Procedure-Related Interaction 

•Gives procedure-related help 

•Gives inadequate procedure-related help 

•Asks a question and receives procedure-related help 

•Asks a question and receives inadequate procedure-related help 

3) Socio-Emotional Interaction 

•Positive socio-emotional interaction 
•Negative socio-emotional interaction 
•Neutral soc ; o-emotional interaction 

4) Miscellaneous Off-Task Interaction 

In MacMagic, all of the above interaction categories were observed. At the beginning of 
each project, there were more task-related and procedure-related interactions among the 
students. These interactions were most common while the students worked at the 
computers. As the project or group activities progressed, more of the socio-emotional 
interactions occurred. Eventually, miscellaneous off-task interactions become more 
common. In the MacMagic classroom, student activities were more structured than in the 
MacMagic Lab. Students were given specific activities to complete within a given time. The 
teachers frequently required everyone's attention during class instruction and during 
discussions. Therefore, there was less socio-emotional and off-task interaction in the 
MacMagic classroom. 

To summarize, in most cases, and especially among students with lower English 
proficiency, students tend to ask questions of someone who speaks their same first 
language. However, due to the emphasis on group activities and peer instruction, most of 
the ESL students (with one or two exceptions) ultimately participated in some form of task- 
oriented interactions with native speakers. ESL students with high English proficiency 
exhibited interaction patterns similar to those of native English speaking students, with 
frequent interaction in all four categories. 
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Did the Interaction Patterns of ESL Students Change Over Time? 



The ESL students' interaction patterns underwent some changes as the school year 
progressed. The "ethnic solidarity," "language solidarity," and "ESL group solidarity" 
remained intact throughout the school year. At the same time, the ESL students increased 
their interaction with native English speakers. Even those few students who'd originally only 
speak to those who shared their native language developed some interactions with native 
English speaking students. 

The seventh grade MacMagic ESL students exhibited clear ethnic and linguistic preferences 
when they interacted. The sixth grade MacMagic ESL students, however, showed a higher 
level of interaction, regardless of ethnic or linguistic background. Towards the end of the 
school year, the pattern of interaction that was observed in the seventh grade students 
started to emerge among the sixth grade ESL students. 

Each individual ESL student, whether in sixth or seventh grade, underwent some changes 
in terms of their interaction patterns (with one exception). Each student became more open 
to both verbal and non-verbal interactions. The students who were very reserved and 
withdrawn at the beginning of the school year changed little by little. By the end of the 
school year they were asking questions and starting conversations. They also responded 
more actively to the native English speakers' questions or comments. A very quiet Asian girl 
said on one occasion, "I don't like to talk to others, because I don't speak good English. 
I am afraid that Americans will make fun of me if I make any mistake." One boy mentioned 
that he does not like to speak English because "when I speak English, they don't 
understand me. I think my pronunciation is not too good. I don't like to repeat the same 
things again and again, because I feel bad. So I feel nervous when I speak English to 
others." By May, both of these students were engaged in verbal interactions with other 
students and teachers much more than at the beginning of the year. 

Were There Any Individual Differences in Language Interaction? 

There were several factors influencing the language interactions of the ESL students in 
MacMagic. The most influential variables were: 

■ Personality: Certain personality traits triggered greater interaction bet- 
ween ESL students and others, especially between the native speakers 
and the teachers. The personality traits included the willingness to take 
risks, lack of inhibition, extroversion and assertiveness. ESL students 
with any of these characteristics tended to interact with others more. 

■ Self-esteem: The ESL students with higher self-esteem tended to ask 
questions and also volunteered answers both in individual and group 
interaction situations. 
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■ Motivation: The ESL students with higher learning motivation or who 
finished given assignments/projects tended to interact more with 
others, to ask questions or to find out better ways of doing their 
assignments. 

■ Familiarity with the people around the student: The ESL students, 
especially those with lower English proficiency, tended to associate 
with the people they knew. 

■ English language (oral) proficiency: The ESL students with higher 
English proficiency tended to interact more with others. They appeared 
to have less anxiety about speaking English. 

How Did the Students Interact With the Computers? 

As a result of computer-integrated instruction, students increased interaction among 
themselves and with the teachers. The interaction of ESL students and the computer in 
MacMagic was particularly interesting. At the beginning of the school year in September, 
most of the ESL students were reluctant and frustrated usirfg the computer, but within a 
week their interest grew. They quickly realized they had control over what they did. Even 
a student whose English proficiency level is low can write, draw, and produce a document 
which looks as good as anyone else's. The computers boosted the students' self-esteem. 
They could learn to do things on the computer all by themselves, sometimes before anyone 
else had. High language proficiency is not required in using the computer. A student with 
limited proficiency can teach and show others how to make a graphic or a HyperCard stack 
work. Contributing and teaching others gives a big boost to the student's self-esteem. 

The students controlled their content as well as their work pace. They seemed to be 
comfortable having control over system parameters and their options to explore the 
computers' effects. By having computers in front of them, the students were able to 
compare their stacks to others, and to try each other's programmed stacks. Naturally, 
comments, questions and explanations were exchanged. 

The Interaction Patterns of the ESL Students in 
MacMagic and ESL Classes 

Do ESL students in MacMagic interact with the native speakers and the teachers 
more than ESL students in ELAC and Sheltered classes? 

The ESL students in MacMagic interacted more with the native English speaking students 
and the teachers than their counterparts in other ELAC and Sheltered classes, perhaps 
because the ESL students in MacMagic had more access to native English speakers. 
Typically, in ELAC or Sheltered classes, the student population is predominately Spanish 
speaking, with few students from other native language backgrounds. The students tend to 
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speak in their native languages if they have the opportunity to do so. This was observed 
among the ESL students in MacMagic as well. They exhibited a clear tendency and 
preference for speaking their native language. 

MacMagic has a low teacher-to-student ratio. The ELAC and Sheltered classes are made 
up of a larger number of students. Also, there are new students entering the class 
throughout the year. In some months, up to three new non-English speaking students are 
enrolled in an ESL class. The teacher-to-student ratio is higher in the regular ESL classes 
than in MacMagic. 

Cooperative learning activities require interaction among students. The ESL classes that 
implemented this cooperative learning approach were quite successful, and resulted in 
student interaction. The classes which didn't use the cooperative approach did not have as 
much interaction. These classes also had more class management and behavior problems, 
and the better the students' English skills, the more they talked to others, the ESL students 
in MacMagic had a higher English proficiency in general. 

Description of Interest/Participation in MacMagic and ESL Classes 

Interest level ran relatively high among the MacMagic ESL students. This issue is related to 
the teachers' classroom management styles, the students' motivation level, and the 
organization of the class. . However, based on the teaching strategies employed by the 
teachers, there were times when the ESL class students were more enthusiastic and 
interested in their learning process than the MacMagic students. 

It's impossible to generalize about the differences between teaching styles in MacMagic and 
other ESL classes. There were seven teachers involved in the observations of five different 
classes with students who had varying language backgrounds and grade levels. Each 
teacher had her own persona! style in the classroom. 

ESL students in the MacMagic classes were given individual attention when necessary. Sixth 
grade MacMagic classes were more structured. They had more instruction sessions where 
the whole class came together and the teachers would lead the class in activities. ESL 
students covered the same subjects as English speaking students through class reading. 
The class was divided into two reading groups. One group was for the native English 
speaking students; the other was for students with lower reading ability. AH but two in this 
group were ESL students. The groups went into two different rooms when they had reading 
group sessions. Typically, the class had to read a certain amount of pages in their reading 
book the night before, and a brief writing assignment was given. In the ESL reading group, 
the teacher would call on each student to read a section of the book aloud. The teacher 
helped students who had difficulty pronouncing some words and corrected mistakes. After 
a student finished reading a section, the teacher would go over the passage, making sure 
it was understood, explaining difficult words in context and in simpler language. 



Chapter V): Student/Teacher Interactions in MacMagic and Regular ESL Classes 

47 



6.7 



For example, when the class finished reading a section of a book called Golden Goblet , the 
teacher would explain what a 'goldsmith' is, and would ask what the main character did 
or what happened in the section. Most of the students participated in the discussions with 
enthusiasm. Interestingly, when the class, wasn't divided, some ESL students never 
participated or volunteered answers to the teachers' questions. However, in their reading 
group, they got involved in the discussion and frequently volunteered answers. The level of 
participation and interaction increased visibly. This was especially obvious with the girts. 
One usually quiet, non-participatory Asian girl became more involved with her reading 
group, improving during the school year, interacting with others and eventually pursuing the 
teachers tc ask questions. As the teachers realized her personal needs and gave her more 
individualized attention, she gained more self-confidence. Towards the end of the school 
year, she was frequently spotted speaking to other students, asking questions, commenting 
on others' computer work, showing her stacks, demonstrating some skills, and asking 
questions of the teachers. 

The teachers in the sixth grade MacMagic class had a tendency to speak in an exaggerated 
manner, pronouncing their words slowly to make their speech clearer. The class was tightly 
managed, yet the students were free to move around and talk to others. The organization 
of the class was well planned in advance; this enabled the teachers to teach efficiently using 
cooperative learning strategies. Two sixth grade MacMagic teachers were often observed 
discussing their new class plans, implementing and adjusting as they went along. The ESL 
students were mostly "mainstreamed" in MacMagic, with access to the same content 
instruction, but were able to receive more individualized help from the teachers and from 
each other. Group discussions and activities encouraged help among the students. The 
groups were carefully organized by the teachers according to each member's ability, 
language background, and factors such as interaction and behavior patterns. They learned 
techniques for analyzing, interpreting, negotiating and communicating their information as 
a team. The noise level in the classroom was rather high when the students were engaged 
in group activities, but the students assumed a higher level of responsibility for learning. 

The sixth graders interacted more freely across gender, ethnic and linguistic lines at the 
beginning of the school year. As time passed, even though the students were more familiar 
with each other, the gender distinction became more obvious. The casual . interaction 
between Spanish speaking girls and boys became less frequent, except among those 
engaged in 'boyfriend-girlfriend* relationships. As the year progressed, the Spanish 
speaking boys became more aware of the importance of their Latino heritage. The Latino 
students were constantly struggling for their identity in the two sometimes confusing and 
conflicting cultures, namely the mainstream/school community and their family/neighbor- 
hood community. During interviews many ESL students stressed the importance of speaking 
Spanish and maintaining their cultural identity. However, at the same time, the students 
faced internal and external pressures to be assimilated into mainstream society. The 
Spanish speaking ESL students usually distinguished themselves as "Latinos," as opposed 
to "Americans." Mostly when faced with the dilemma, the Spanish speaking students, 
especially the boys, emphasized their Latino heritage by speaking Spanish. 
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Seventh grade MacMagic classes were different in terms of students 1 interactions and class 
organization, more relaxed with more student freedom. The seating arrangement in the 
MacMagic classroom changed several times throughout the school year. Because of the 
emphasis on cooperative learning the students were seated in groups all year long. Every 
activity in the class included at least some group work or cooperation among the students. 
The students were encouraged to help each other, consult, and critique each other's work. 
With the exception of two more advanced students, the ESL students were separated from 
the rest of the class for the "word bank" session, when they would go over words and 
grammar rules (mostly mechanics and punctuation). ESL students were assigned basic 
study words, in addition to the native speaker's pre-advanced vocabulary. The ESL group 
was tested separately on the "word bank" quiz day. The teachers gave individualized help 
to any student when it was needed. Frequently, the teachers helped ESL students to catch 
up after school. 

The ELAC 2 and Sheltered classes were taught by three different teachers with different 
teaching styles. These ESL teachers had classes of 20 to 40 (depending on the class) 
students learning English as their second language. All ESL classes had some form of 
grammar instruction and vocabulary instruction sessions. Grammar and vocabulary studies 
are an important part of second language learning. In classrooms with many students, some 
speaking different native languages, such study can be difficult: many of the students don't 
know the basic rules of the language. After grammar and vocabulary instruction sessions, 
the students were given quizzes and assigned homework related to the lesson. 

In order to provide grade level content instruction, teachers employed the strategies of 
"sheltered instruction," which involves using a variety of descriptions so the student will 
understand the content. The teachers explained the words and concepts in each lesson 
using pictures from books, overhead projector images and video tapes. 

Some of the ESL classes also implemented cooperative learning groups, activities and 
projects. The students were seated in groups of four or five. These were very successful 
in terms of encouraging meaningful learning interactions. The students responded to the 
teachers' lessons with enthusiasm and interest. Helping each other and working together 
was part of their everyday class activities. The ESL classes not employing group work or 
cooperative approaches spent more time on student behavior problems. In these classes, 
the students sat in rows in the same seating arrangement throughout the school year. 
Typically, talk among students was discouraged. Getting together for in-class assignments 
was not the norm. Therefore, students exhibiting a tendency to work together was viewed 
as negative behavior. 

Table 6.1 outlines the differences between MacMagic and ESL classes. 
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Table 6.1: Differences Between MacMagic and ESL Classes 



Teachers 

I.A (Instructional Assistant) 
Other Resource 

Technology 

Students (number) 
Language background 
ESL Number 

Class time 



Organization 



MacMagic 
2 in each class 
0 

One full-time technical support person 
Student TAs 

Multimedia computer lab (with computers, 
laser printer, scanner, laserdisc player, 
computer screen projector, VCR, video 
camera ...) 

28 per class 



ESL classes 

t in each class 
1* 



Overhead projector 
1 Mac Classic" 



20 to 40 per class 



Mixed, a majority of native English speakers All ESL 



22-27 out of 1 12 (1-10 out of 28 to 30 
students per class) 

3 periods everyday 



stayed together all year 

Cooperative learning (used extensively) 

-peer editing 

•peer instruction 

-group projects & presentations 

Individual assignments 

-reading 

•writing (with hand and word processed) 
•process writing using word processor 
-other works 

Content area instruction (CA framework- 
based) 

Individualized attention 

Portfolio assessment (process assessment) 

Technology facilitated 

Hands-on activities 

ESL students mainstreamed 

Relaxed, students walking around freely 

ESL students spoke their first language with 

other ESL students who spoke the same 

language 



whole class 



3 periods everyday - 6th ELAC class 

3 periods everyday - 7th & 8th combination 

ELAC class 

2 periods everyday - 7th Sheltered Core 

received new students throughout the year 

Cooperative learning (depends on the 
teacher) 
-group projects 
-group activities 

-hands-on activities (depends on the teacher) 
-group writing (depends on the teacher) 
Individualized assignments 
•reading 

•writing (hand written) 
-other works (vocabulary, grammar) 
Grammar, rules, usage, mechanics 
separately taught 

Individualized attention (when the teacher 
can offer H) 

Assessment (part portfolio and part test and 
quiz scores) 

Content area instruction (CA frame work) 

ESL students taught in sheltered language 

Whole language approach relaxed 

ESL students spoke their first language with 

other ESL students who spoke the same 

language 



* 7th grade Sheltered class did not have an instructional assistant every day. This class only had an 
Instructional assistant for 30 to 40 minutes three days a week. 

** Not all non- Mac Magic classes had a computer in the classrooms. 7th grade Sheltered class did not have 

a computer in the classroom. 
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Chapter VII: 
Findings from the 

Comparison of MacMagic Students 

to Regular ESL Students === 

In the following sections, we present the results of quantitative analyses designed to test 
several specific hypotheses. First, we look briefly at attendance figures. Then we present 
the results of several repeated-measures Analyses of Variance (ANOVAs) which test for 
differences between MacMagic and ESL control groups on an array of objective measures. 

Comparison of the Absence Rates of the MacMagic and ESL Groups 

The average days absent per student are presented in Table 7.1. As shown, the lowest 
absence rate occurred among sixth grade MacMagic students while the highest absence 
rate occurred among seventh grade ESL comparison students. 

Table 7.1: Mean Absence Rates for MacMagic and 
ESL Students in 6th and 7th Grades 





6th 


7th 


MacMagic 


3.286 


6.600 




n=7 


n=15 




sd=3.638 


Sd= 6.620 


ESL 


5.111 


9.353 




n=9 


n=17 




sd =5.840 


Sd = 9.353 



A 2 x 2 Analysis of Variance (ANOVA) was used to examine absence rates among ESL and 
MacMagic students in sixth and seventh grade. There was a nonsignificant trend toward 
fewer absences in sixth grade, compared to seventh grade, F(1,44) = 2.86, p = .10. As 
shown in Table 7.1, sixth graders averaged just over four absences, while seventh graders 
averaged just over eight absences. The difference between MacMagic students and ESL 
students did not approach statistical significance, F(1,44) = 1.05, p = .31, nor did the 
interaction between grade and condition, F(1, 44) = .043, p = .83. 
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The Relationship Between the Absence Rates and 
Language Proficiency Measures 

In order to investigate the influence of the number of absences on various language 
proficiency scores, simple correlations were run between number of absences and all 
dependent measures, including pre-test scores, post-test scores, and change scores on 
each measure. Change scores were generated for each student on each measure by 
subtracting pre-test scores from post-test scores. 

There was a clear relationship between the pre-test scores and the post-test scores: the 
higher the pre-test score, the higher the post-test score. However, the correlational analyses 
yielded no other significant results, suggesting that number of absences did not have any 
measurable relationship with students' initial ability levels, with their final ability levels, nor 
with their improvement on the variables examined in the current study. 

Language Proficiency Scores 



Table 7.2: Mean Pre-test and Post-test Language Proficiency Scores 



A 2 x 2 ANOVA was used to examine changes in language proficiency scores for 
MacMagic and ESL students. Pre-test and post-test proficiency scores were a within 
subjects variable, while conditio.": (MacMagic versus ESL) was a between subjects 
variable. 

There was a nonsignificant trend toward higher scores for MacMagic students, relative 
to ESL students, F(1,46) = 3.18, p = .08. As shown in Table 7.2, MacMagic students' 
scores v/ere slightly higher at both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1 ,46) = 50.24, p < 
.0001. As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 

The interaction between condition (MacMagic versus ESL) and testing time (pre-test 
versus post-test) proved nonsignificant, F(1,46) = .006, p = .67, suggesting that there 



Pre-test 



Post-test 



MacMagic 

Mean = 41 6.364 

n = 22 

sd = 74.990 

Mean = 454.364 

n = 22 

sd = 59.390 



ESL 

Mean = 383.654 

n = 26 

sd = 75.169 

Mean = 420.846 

n = 26 

sd = 54.782 
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was no difference between MacMagic and ESL students in language proficiency score 
improvement. 

To summarize, both MacMagic students and ESL group students improved about 
equally over the course of the year. 

Reading/Writing Total Scores 

Table 7.3: Mean Pre-test and Post-test Reading/Writing Total Scores 



Reading and writing scores were added for each student to yield a combined 
reading/writing score. A 2 x 2 ANOVA was used to examine changes in reading/writing 
scores for MacMagic and ESL students. Pre-test and post-test reading/writing scores 
were a within subjects variable, while condition (MacMagic versus ESL) was a between 
subjects variable. 

There was a nonsignificant trend toward higher scores for MacMagic students, relative 
to ESL students, F(1,46) = 1.98, p = .17. As shown in Table 7.3, MacMagic students' 
scores were slightly higher at both pre-test and post-test. There was a significant main 
effect for pre-test versus post-test, F(1,46) = 35.95, p = .0001. As would be expected, 
students' scores were significantly higher at the time of the post-test, than at the time 
of the pre-test. 

The interaction between condition and testing time proved nonsignificant, F(1,46) = 
.187, p = .67, suggesting that there was no difference between MacMagic and ESL 
students in reading/writing score improvement. 
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Pre-test 



Post-test 



MacMagic 

Mean = 128.864 

n = 22 

sd = 25.791 

Mean = 143.045 

n - 22 

sd = 21.217 



ESL 

Mean = 118.846 

n o 26 

sd = 24.906 

Mean = 135.231 

n = 26 

sd = 22.201 
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Reading Scores 



fable 7.4: Mean Pre-test and Post-test Reading Scores 



Pro-test 



Post-test 



MacMagic 

Mean = 68.636 

n = 22 

sd = 14.774 

Mean = 73.455 

n = 22 

sd = 14.774 



ESL 

Mean = 64.462 

n = 26 

sd = 13.935 

Mean = 70.615 

n = 26 

sd = 16.337 



A 2 x 2 ANOVA was used to examine changes in reading scores for MacMagic and ESL 
students. Pre-test and post-test reading scores were a within subjects variable, while 
condition (MacMagic versus ESL) was a between subjects variable. 

The main effect for condition did not approach significance, F(1,46) = 0.78, p = .38. 
As shown in Table 7.4, MacMagic students ' scores were slightly but non-significantly 
higher than ESL students' at both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1,46) = 8.69, p = 
.0051. As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 

The interaction between condition and testing time proved nonsignificant, F(1,46) = 
.129, p = .72, suggesting that there was no difference between MacMagic and ESL 
students in reading score improvement. 

Writing Scores 



Table 7.5: Mean Pre-test and Post-test Writing Scores 



Pre-test 



Post-test 



MacMagic 

Mean = 60.227 

n = 22 

sd = 13.32 

Mean = 69.591 

n = 22 

sd = 10.051 



ESL 

Mean = 54.385 

n = 26 

sd = 13.455 

Mean = 64.615 

n = 26 

sd = 8.6073 
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A 2 x 2 ANOVA was used to examine changes in writing scores for MacMagic and ESL 
students. Pre-test and post-test reading scores were a within subjects variable,, while 
condition (MacMagic versus ESL) was a between subjects variable. 

There was a nonsignificant trend toward higher scores for MacMagic students, relative 
to ESL students, F(1,46) = 3.438, p = .07. As shown in Table 7.5, MacMagic 
students' scores were slightly higher at both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1 ,46) = 36.36, p = 
.0001. As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 

The interaction between condition and testing time proved nonsignificant, F(1,46) = 
.071 , p = .79, suggesting that there was no difference between MacMagic and ESL 
students in writing score improvement. 

Oral Total Scores 



A 2 x 2 ANOVA was used to examine changes in oral total scores for MacMagic and 
ESL students. Pre-test and post-test oral scores were a within subjects variable, while 
condition (MacMagic versus ESL) was a between subjects variable. 

There was a nonsignificant trend toward higher scores among MacMagic students, 
relative to ESL students, F(1,46) = 3.16, p = .08. As shown in Table 7.6, MacMagic 
students' scores were slightly higher at both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1,46) =21.01, p = 
.0001 . As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 



Table 7.6: Mean Pre-test and Post-test Oral Total Scores 



Pre-test 



Post-test 



MacMagic 

Mean = 72.955 

n = 22 

sd = 15.085 

Mean = 78.864 

n = 22 

sd = 13.130 



ESL 

Mean = 67.385 

n = 26 

sd = 13.606 

Mean = 71 .423 

n = 26 

sd = 10.871 
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The interaction between condition and testing time proved nonsignificant, F(1,46) = 
.745, p = .39 ( suggesting that there was no difference between MacMagic and ESL 
students in oral score improvement. 



Essay Scores 



Table 7.7: Mean Pre-test Post-test Essay Scores 



Pre-tost 



Post-test 



MacMagic 
Mean = 3.000 
n = 22 
sd = .690 
Mean = 3.523 
n = 22 
sd = .523 



ESL 

Mean = 2.692 
n = 26 
sd = .679 
Mean = 3.17 
n = 26 
sd = .399 



A 2 x 2 ANOVA was used to examine changes in essay scores for MacMagic and ESL 
students. Pre-test and post-test essay scores were a within subjects variable, while 
condition (MacMagic versus ESL) was a between subjects variable. 

There was a significant main effect for condition, F(1 ,46) = 5.38, p = .02. As can be 
seen in Table 7.7 MacMagic students' scores were significantly higher than ESL 
students' at both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1,46) = 29.87, p = 
.0001. As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 

The interaction between condition and testing time proved nonsignificant, F(1,46) = 
.006, p = .67, suggesting that there was no difference between MacMagic and ESL 
students in essay score improvement. 
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Table 



7.8: Mean Pre-test and Post-test Essay Length 



Pre-test 



Post-test 



MacMagic 

Mean = 93.000 

n = 22 

sd = 48.694 

Mean = 125.227 

n = 22 

sd = 36.731 



ESL 

Mean = 73.115 

n = 26 

sd = 38.883 

Mean = 118.731 

n = 26 

sd = 34.952 



A 2 x 2 ANOVA was used to examine changes in essay length for MacMagic and ESL 
students. Pre-test and post-test essay length scores were a within subjects variable, 
while condition (MacMagic versus ESL) was a between subjects variable. 

The main effect for condition was not significant, F(1,46) = 1.63, p = .21. As shown 
in Table 7.8, MacMagic students' scores were slightly but non-significantly higher at 
both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1,46) = 55.614, p < 
.0001. As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 

The interaction between condition and testing time proved nonsignificant, F(1,46) = 
1.651, p = .21, suggesting that there was no difference between MacMagic and ESL 
students in essay length change. 

Oral Language Sample Scores 

Table 7.9: Mean Pre-test and Post-test Oral Language Sample Scores 



Pre-test 



Post-test 



MacMagic 
Mean = 3.455 
n = 22 
sd = .963 
Mean = 3.886 
n = 22 
sd = .689 



ESL 

Mean = 3.077 
n = 26 
sd = .744 
Mean = 3.500 
n = 26 
sd = .600 
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A 2 x 2 ANOVA was used to examine changes in oral language sample scores for 
MacMagic and ESL students. Pre-test and post-test essay length scores were a within 
subjects variable, while condition (MacMagic versus ESL) was a between subjects 
variable. 

There was a marginally significant main effect for condition, F(1 ,46) = 3.79, p = .06. 
As shown in Table 7.9, MacMagic studonts' scores were higher than ESL students' at 
both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1,46) = 20.04, p = 
.0001. As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 

The interaction between condition and testing time proved nonsignificant, F(1,46) = 
0.002, p = .96, suggesting that there was no difference between MacMagic and ESL 
students in oral language sample score improvement. 

Oral Language Sample Length 

Table 10: Mean Pre-test and Post-test Oral Language Sample Length 



A 2 x 2 ANOVA was used to examine changes in oral language sample length for 
MacMagic and ESL students. Pre-test and post-test essay length scores were a within 
subjects variable, while condition (MacMagic versus ESL) was a between subjects 
variable. 

There was a nonsignificant trend toward higher scores by MacMagic students, relative 
to ESL students, F(1 ,46) = 3.02, p = .09. As shown in Table 7.10, MacMagic students' 
scores were slightly higher than ESL students' at both pre-test and post-test. 

There was a significant main effect for pre-test versus post-test, F(1 ,46) = 37.45, p = 
.0001 . As would be expected, students' scores were significantly higher at the time of 
the post-test than at the time of the pre-test. 



Pre-test 



Post-test 



MacMagic 

Mean = 107.091 

n = 22 

sd = 41 .023 

Mean = 135.227 

n = 22 

sd = 36.487 



ESL 

Mean = 84.346 

n = 26 

sd = 37.235 

Mean = 122.615 

n = 26 

sd = 32.629 
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The interaction between condition and testing time proved nonsignificant, F(1,46) = 
1.61, p = .21, suggesting that there was no difference between MacMagic and ESL 
students in oral language sample length change. 



Conclusions 

The results presented in this chapter suggest the following conclusions: 

■ The results of the quantitative analyses suggest no differences 
between MacMagic students and ESL control students in 
improvement on the variables examined in the current study. 

■ It is worth noting that MacMagic and ESL students' initial ability 
levels were not perfectly matched: MacMagic students ' scores were 
higher on most measures at the time of the pre-test, and they 
remained so at the time of the post-test. However, there were 
apparently no differential effects of the MacMagic and ESL 
programs. 

■ Had there been trends toward greater improvement among 
MacMagic (or ESL control) students, their higher initial scores might 
be cause for concern, as the so-called "ceiling effect" might lessen 
the degree of improvement possible among MacMagic students. 
However, the ceiling effect is probably not responsible for the lack 
of a program effect in the current study, given the lack of even a 
slight trend in the direction which would be predicted by such an 
effect. 

■ There were slightly fewer absences among sixth graders than 
among seventh graders. However, the difference between 
MacMagic students and ESL controls in number of absences did 
not approach significance. Surprisingly, numbers of absences 
weren't related to any of the performance measures examined in 
the current study. 
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Chapter VIII: 

Conclusion and Comments 



Our year-long study on second language acquisition by ESL students in the MacMagic 
program leads to the following conclusions: 

■ The MacMagic group and the ESL group achieved about the same 
language growth in the 1992-'93 school year regardless of their core 
programs. 

■ The amount of social interaction between ESL students and native 
English speakers was higher in MacMagic. 

■ MacMagic facilitates English as a second language acquisition among 
ESL students as effectively as any other ESL classes such as ELAC or 
Sheltered classes which emphasize English language acquisition. 

■ In addition to English language acquisition, the students in MacMagic 
acquired additional computer skills, the ability to process abstract 
concepts, and socialization skills involved in the cooperative learning 
process, with computers as facilitative tools. 

Acquisition of English as a second language involves many different variables, including 
attitude, motivation, aptitude, home situation, socialization, native language background, 
length of residence, and exposure to the target language environment. In this study, the 
variable examined was exposure to the MacMagic program. Classroom observations of 
student language interactions, interviews with students concerning MacMagic, and English 
language proficiency scores from the LAS (Language Assessment Scale) were the primary 
data sources for the investigation. 

Results revealed that although ESL students did learn English in MacMagic, they did not 
acquire English significantly faster or with greater proficiency than ESL students in non- 
MacMagic classes. However, the English proficiency scores don't capture the total effect 
MacMagic had on the students. One must consider the process as well as the product. 
In this study, the observational and interview results suggested differences in the 
experiences of MacMagic and non-MacMagic students. 

Students in MacMagic used three software packages to create their own work. Most wrote 
on the word processor and enhanced their writing with graphics. They also created their 
own HyperCard stacks. In order to create a HyperCard stack, students conducted 
research and organized information using associative thinking skills. We recommend that 
in future studies, researchers attempt to more objectively measure MacMagic's effect on 
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students 1 use of associative thinking skills, as well as MacMagic's effect on self-esteem, 
socialization skills and computer literacy. Even with their limited English proficiency, ESL 
students were able to participate in class activities and projects with other native English 
speakers. The teaching staff was especially accessible to ESL MacMagic students. The 
availability of an extra teacher, a computer technician, student teaching assistants, native 
speaking students and other ESL students, all contributed to the learning environment. 

As noted, both classroom observation and interviews suggested that reaction to the 
MacMagic program was generally positive. Students felt in charge of their own learning and 
seemed to benefit from having teachers function as facilitators. Collaborative learning and 
process (portfolio) assessment also contributed to a positive and unique learning 
environment. 

To better understand the effects of computer facilitated programs, further research should 
be done on the long-term benefits of MacMagic and similar programs on such aspects of 
learning as motivation, attitude, and socialization of the students. Innovative curricula as well 
technology-based instructional methods are among many possible second language 
learning schemes. 

We conclude with a brief discussion of ways in which we think MacMagic can be improved. 
Areas for Improvement ' 

In general, classroom observations and student interviews have painted a positive picture 
of MacMagic. Students seemed to enjoy it, and the learning aspects were successful. Still, 
we would like to suggest several ways in which MacMagic, and similar programs, can be 
improved in the future. 

■ Strive to make the relationship between computer-based projects and 
the content area (e.g., language arts or social studies) as clear as 
possible. There were several instances in which students spent a 
great deal of time creating HyperCard stacks without using the 
experience to iearn content-specific information. 

■ Strive to maximize the frequency of interaction between students of 
different languages and cultures. In spite of the opportunity for such 
cross-cultural communication, there was a tendency by students to 
seek out members of their own cultural or language background. 

■ Ensure that teachers are comfortable around computers and able to 
use them to their fullest extent. 

■ Increase student accountability. The relaxed, conversational atmos- 
phere present in MacMagic allowed some students to "goof off." 
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Although most students used their freedom in a positive way, there 
were several who took advantage of the situation and did very little 
work. 

Explicitly address multi-cultural and multi-ianguage-related issues to a 
greater degree in the curriculum, if possible. One suggestion would 
be to directly address cultural differences interactively, both in teacher 
training and in the curriculum. 
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